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(57) Abstract 



A controller, in accordance with a preferred embodiment, provides, in a specified region of first television picture generated by a 
first digitally compressed video signal, an overlay of a second picture embodied in a second video signal. The controller includes an 
I-frame buffer for storing updated macroblock pixel data in I-frame format derived from the second signal. The controller also has a matte 
mask storage register for storing data that identifies the specified region by means of macroblocks. A matte substitution processor is in 
conmiunication with the matte mask storage register and has as an input the first signal and provides an overiayed output; the controller 
inhibits transmission to the output of any macroblock, of the first signal's data stream, lying in the specified region. It also functions to 
substitute for such macroblock a corresponding macroblock from the I-frame buffer. Further embodiments permit overlay of a graphic 
display for interactive television on a picture derived from a digitally compressed video signal- 
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COMPRESSE D DIGITAL QVERIAY cnKrronT.T.T!P AND METHOD 

5 Teclmleal Field 

The present invention relates to controllers for 
providing in a first television picture generated by signals 
in a compressed digital format, such as under the MPEG2 
standard, an overlay of a second picture. 

10 Baclcaround Art 

Overlay controllers for analog television signals are 
well known in the art* The MPEG 2 compression standard for 
digital video signals also contemplates the possibility of 
overlaying a first compressed digital video signal with a 

15 second compressed digital video signal* A controller for 

achieving the overlay of two such digitally compressed video 
signals is inherently complex* In some instances, it is 
desiraijle to overlay a compressed digital video signal (such 
as, for exsuaple, a movie or other material made avail2Q>le to 

20 a csJdle television subscriber) with a relatively slowly 
changing analog signal (such as, for example, a display 
including a movable cursor and a nxombeir of selection buttons 
to permit interactive caible television servicei to a 
sxibscriber operating a remote control unit causing movement 

25 of the cursor and operation of the buttons) . In such a 

case, the art currently would require, before overlaying the 
signal, either (i) A-to-D conversion of the analog signal 
and its subsequent compression or (ii) decompression and D- 
to-;-A conversion of the digital signal. Such techniques are 

30 inefficient and expensive. 

Bnmmnrv of the Invention 
In accordance with a preferred embodiment of the 
invention there is provided a controller, for providing, in 
a specified region of first television picture generated by 

35 a first digitally compressed video signal, an overlay of a 
second picture embodied in a second video signal. The 
controller includes an I-frame buffer for storing updated 
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macroblock pixel data in I-frame format derived from the 
second signal. The controller also has a matte mask storage 
register for storing data that identifies the specified 
region by means of macroblocks. A matte sxibstitution 
5 processor is in commianication with the matte mask storage 
register and has as an input the first signal and provides 
an over lay ed output; the controller inhibits transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region. It also functions to 

10 substitute for such macroblock a corresponding macroblock 

from the I-frsune buffer. Further embodiments permit overlay 
of a graphic display for interactive television on a picture 
derived from a digitally compressed video signal. 

Briief Description of the ipravinas 

15 The inventidn will be more readily iinder stood by 

reference to the following descriptidn taken with the 
accompanying drawings in which: 

Fig. 1 is a diagreun of the effect of aun overlay in 
accordance wiUi a preferred embodiment of the invention? 

20 Fig. 2 is a diagrsun of a matte mask used to define the 

boundaries of the overlay in Fig. 1; 

Fig. 3 is a block diagram of an embodiment of the 
invention for achieving the overlay of Fig. 1; and 

Fig. 4 provides detail of certain portions of Fig. 3. 

25 Detailed Description of Specific Embodiment a 

Fig. 1 illustrates the effect of an overlay of a first 
MPEG2 encoded digital video source, which is referred to 
below as source 1, by a second signal, which is referred to 
below as source 2 . In the context of interactive 

30 television, source 1 may be, for example, a digitally 

compressed movie or other information service (as that term 
is defined above). Source 2 may be, for example, a graphic 
display, in analog format, showing a menu of choices 
available to a subscriber and a cursor movable by the 

35 subscriber (using a remote control), so arranged as to 

permit the subscriber to communicate interactively with the 
headend and to make a selection from the menu of choices. 
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Thus, source 1 could be a movie preview, for example, and 
source 2 could present a menu of choices to the subscriber 
permitting selection of the movie or an opportunity to 
preview another movie, and so oh. In this example, source 1 
5 is shown in region A of Fig. 1 and source 2 is shown in 
region B. 

In accordance with MPEG2 encoding, successive 
contiguous 16 X 16 square arrays (herein called 
"macroblocks") of pixels are identified over the entire 

10 picture area; a sample macroblock is shown as item 11 in 
Fig. 1. Each macroblock in the picture area is given a 
uniqpie nimerical address, in the general manner illustrated 
in Fig. 1, corresponding to its location in the picture. 
The boundary between regions A and B in Fig. 1 is 

15 defined by a matte mask, shown in Fig. 2, that spetcifies the 
macroblock addresses that are included (or, alternatively, 
excluded) in region A. 

Fig. 3 is a block diagram for an embodiment of the 
present invention for achieving the overlay described above. 

20 Inputs 30 include the digitally compressed source 1 signal 
as well as data showing activity of the subscriber's remote 
unit. (Additional video data may be provided for use in 
creating the video for region B of Fig. 1. Such additional 
data may come as digitally compressed data from the inputs 

25 30 aind be decompressed by decompressor 33, and/ or it may 

come from other source 34, which may be a decompressor or TV 
tuner. Data from the subscriber's remote unit is prbcesised 
by CPU 31, operating with RAM 35, and graphics overlay 32 
provides an analog input to mixer-encoder 36. This input 

30 321 shows cursor position and the menu choices available to 
the subscriber for interaction with the headend. The mixer- 
encoder 36 first mixes input 321 with inputs from sources 33 
and 34, and then encodes the mixed analog signal as a series 
of I-frames in MPEG2 format. The I-frame encoded output of 

35 mixer-encoder 36 is provided to MPEG2 mixer 37, the output 
of which is subjected to regulation by flow controller 38 to 
prevent excessively high data rates from appearing at the 
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output 39. This regulation is necessary to prevent the data 
rates from exceeding the capacity of the distribution 
system/ and, particularly, of the subscriber's equipment. 

Detail of the mixer-encoder 36, the MPEG2 mixer 37, and 
5 the flow controller 38 is presented in Fig. 4. The MPEG2 
encoded digital video data stream identified as source 1 in 
Fig. 4 is here overlayed with a desired signal output, 
identified as source 2, from the video effects and mixer 
module 41. in the manner described herein. In this exeunple, 

10 source 2 is in analog YUV format. The source 2 signal is 
then digitally encoded by encoder 42 as a series only of I- 
frames in MPEG2 format. (That is, no B-frames or P-frames 
are provided in the output from encoder 32. It will be 
appreciated that the picture content of region B is 

15 geherally expected to be rela^^ slowly changing; the / 

changes most commonly expected are primarily those 
associated with cursor motion and selection of choices by 
the subiscriber. Accordingly, the full MPEG encoding of 
source 2 is not necessary. 

20 Mpreover^ constraints x>n the system are required in 

order for it to provide appropriate responsiveness. Ejach 
freune duration is l/30th of a second. Encoding source 2 as 
a series of I-frctmes will therefore mean up to l/30th of a 
second until an action on the subscriber's remote unit 

25 (excluding other sources of delay) will have been encoded by 
encoder 42. Moveover, in addition to the time for encoding 
here by encoder 42, there must also be taken into account an 
equal time for decoding for display on the subscriber's 
television; thus the total delay is double the time for 

30 encoding. It is possible, for example, to use P freunes 

along with I frcimes, but if P frames are used in addition to 
I frames, the encoding time will be doubled. 

The I-fraune output of encoder 42 is stored in I-freune 
buffer 33, the data in which is updated as necessary based 

35 on output from the encoder 42. Any of the macroblocks in 
the I-fraone buffer that do not change do not need to be 
updated, cind where there is little change in source 2 video 
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there will be relatively infrec[uent updating of the buffer 
42, 

The I-frame data in buffer 43 is selectively inserted 
in the source 1 HPEG2 encoded data stream in the following 
5 manner. The source 1 data stream is run through macroblock 
detector 46 to detect macroblocks, and its output is 
provided to the matte substitution processor 44 . Matte mask 
storage register 45 stores the addresses of the macroblocks 
constituting the matte mask of Fig. 2, and provides data to 

10 the matte processor 44. Whenever the processor identifies a 
macroblock (regardless whether or P-f ramie) from 

detector 46 that lies outside (or, altemativ^iy, inside) 
the matte region defined in register 45, it substitutes an 
I-frame macroblock of the corresponding address from I-frame 

15 buffer 43. 

It is possible^. if there is a great deal of change in 
the picture content of source 2, then the output 39 of 
system would be at a data rate beyond system capacity. In 
such a case, the processor 44 will continue to block source 

20 1 macroblocks lying outside the matte region, but will 

refrain from substituting data from the I-frame buffer 43 to 
the extent necessary to slow the data rate at the output to 
a rate within system limits. 
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What: is claimed is: 

1. A controller, for providing, in a specified region 
of a television picture generated by a first digitally 
compressed video signal, an overlay of a second picture 

5 embodied in a second video signal, comprising: 

an I- frame buffer for storing updated macroblock pixel 
data in I-frame format derived from the second signal; 

matte mask storage register for storing data that 
identifies the specified region by means of macroblocks; 
10 matte substitution processpr, in communication with the 

matte mask storage register and having as ah input the first 
signal and providing an over 1 ay ed output, for ii^ibiting 
transmission to the output of any macroblock, of the first 
signal's data stream, lying in the specif ied regioii and for 
15 substituting for such macroblock a corresponding macroblock 
from the I-frame buffer. 

2. A controller according to claim l, wherein the 
second video signal is in an analog format, further 
comprising: 

20 an I^frame encoder for encoding the second signal as a 

serieis of I-frcunes, the I- freuae encoder having ah output in 
coxamuhication with the I-frabne buf fer. 

3. A controller according to claim 2, further 
comprising: 

25 a macroblock detector in communication with the first 

signal and the matte siibstitution processor, for detecting 
the presence of a macroblock. 

4. A controller according to claim 1, wherein the 
matte substitution processor includes flow control means for 

30 refraining from substituting data from the I-frame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 
overlayed output to a rate within system limits. 

35 5. A controller according to claim 3, wherein the 

matte substitution processor includes flow control means for 
refraining from substituting data from the I-freune buffer. 
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while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 
overlayed output to a rate within system limits. 
5 6. An interactive television system providing, in a 

specified region of a television picture; generated by a 
first digitally compressed video signal, an overlay of a 
graphic display permitting user inpat via a remote control, 
such graphic display embodied in a second signal in an 
10 analog format, the system cdmpr is Ing: 

an I-frcone encoder for encoding the second signal as a 
series of I-frames; 

cm I-freimis buffer for s macroblock pixel 

data in I-f rame fbzrmat f rbm the I-f rame encoder; 
15 taatte mask stdragi^ r^^lster for storing data that 

identifies the specified region by means of macroblocks; 

matte substitution processor, in communication with the 
matte mask storage register and having as an input the first 
sigmal and providing an oyeriayed output > for inhibit 
20 transmission to.the output of any macroblock, of the first 
signal's data stream, lying in the specified region and for 
substituting for such macroblock a corresponding macroblock 
from the I-f rame buff er. 

?• A controller according to claim 6, further 
25 comprising: 

a macroblock detector in communication with the first 
signal and the matte substitution processor, for detecting 
the presence of a macroblock* 

8. A controller according to claim 6j wherein the 

30 matte substitution processor includes flow control meems for 
refraining from substituting data from the I-f rame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specified 
region, to the extent necessary to slow the data rate at the 

35 overlayed output to a rate within system limits. 

9. A controller according to claim 7, wherein the 
matte substitution processor includes flow control means for 
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refraining from substituting data from the I-frame buffer, 
while continuing to inhibit transmission of any macroblock 
of the first signal's data stream lying in the specif ied 
region, to the extent necessary to slow tiie data rate at the 
5 over! ayed output to a rate within system limits. 

io. A method of overlaying in a specif ied region of a 
television picture a first digitally compressed video signal 
with a second analog video signal, comprising: 

encoding the second signal as a series of X-^frames; 
10 storing the encoded I-frame macroblock pixel data in an 

i-frame buffer; 

storing data that identifies the speGif led region by 
means of macroblocks in a matte mask storage rieglster; 

processing the data stream of the first signal and 
15 providing an overlayed output, by iiihibitihg transmiission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region and substituting for 
such macroblock a corresponding macroblock from the I-frame 
buffer. . ;■ 

20 11. A method according to claim 10, furtheir 

cbis^rising: 

refraining from substituting data from the I-frame 
buffer, while continuing to inhibit transmission of any 
macroblock of the first signal's data stream lying in the 
25 specified region, to the extent necessary to slow the data 
rate at the overlayed output to a rate within system limits. 

12. A method of providing an interactive television 
system having, in a specified region of a television picture 
generated by a first digitally compressed video signal, ah 
30 overlay of a graphic display permitting user input via a 
remote control, such graphic display embodied in a second 
signal in an analog format, the method comprising: 

encoding the second signal as a series of I-frames; 

storing the encoded I-frame macroblock pixel data in an 
35 I-frame buffer; 

storing data that identifies the specified region by 
means of macroblocks in a matte mask storage register; 
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processing the data stream of the first signal emd 
providing am overlayed output, by inhibiting transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified regioh and substituting for 
5 such macroblock a corresponding macroblock from the I— frame 
buffer. ■ 

13. A method according to claim 12, further 
comprising: 

refraining from substituting data from the I-frame 
10 buffer, while continuing to inhibit transmission of any 
macroblock of the first signal's data stream lying in the 
specified region, to the extent necessary to slow the data 
rate at the overlayed output to a rat4 i within ^ s^^ 



15 
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